Method and apparatus for encoding and decoding image by using large transform unit

ABSTRACT

A method of decoding an image including performing entropy-decoding to obtain quantized transformation coefficients of at least one transformation unit in a coding unit of the image, performing inverse-quantization and inverse-transformation on the quantized transformation coefficients of the at least one transformation unit to obtain residuals, and performing inter prediction for at least one prediction unit in the coding unit to generate a predictor and restoring the image by using the residuals and the predictor.

CROSS-REFERENCE TO RELATED PATENT APPLICATION

This application is a continuation of U.S. application Ser. No.13/006,652 filed on Jan. 14, 2011, in the U.S. Patent and TrademarkOffice, which claims priority from Korean Patent Application No.10-2010-0003558, filed on Jan. 14, 2010, in the Korean IntellectualProperty Office, the disclosures of which are incorporated herein byreference in their entireties.

BACKGROUND

1. Field

Exemplary embodiments relate to a method and apparatus for encoding anddecoding an image, and more particularly, to a method and apparatus forencoding and decoding an image by transforming a pixel domain image tocoefficients of a frequency domain.

2. Description of the Related Art

In most methods and apparatuses for encoding and decoding an image, animage of a pixel domain is transformed to a frequency domain and thetransformed image is encoded to compress the image. Discrete cosinetransform (DCT) is a well-known technology used to compress audio/video(AV) data. In recent years, many attempts to find more efficientencoding methods have been made. In audio coding, parametric codingperforms better than DCT and, in two-dimensional data, Karhunen Loevetransform (KLT) has a minimum bit size but has a large overhead size.

SUMMARY

Exemplary embodiments provide a method and apparatus for encoding anddecoding an image by using effective discrete cosine transform (DCT),and a computer readable recording medium having recorded thereon acomputer program for executing the encoding and decoding.

According to an aspect of an exemplary embodiment, there is provided amethod of encoding an image, the method including: performing predictionon a plurality of coding units of the image and generating a pluralityof prediction units based on the predicted plurality of coding units;grouping the plurality of prediction units into a transform unit;transforming residual values included in the grouped plurality ofprediction units into a frequency domain, based on the transform unit,into frequency component coefficients of the frequency domain;quantizing the frequency component coefficients; and entropy-encodingthe quantized frequency component coefficients.

The grouping may include grouping comprises grouping the plurality ofprediction units based on depths of the plurality of prediction unitsthat indicate a degree of hierarchically decreasing a maximum codingunit to the plurality of coding units.

The grouping may include grouping comprises selecting adjacentprediction units among the plurality of prediction units on whichprediction is performed according to a type of prediction mode.

The performing prediction may include generating residual values of theplurality of coding units by intra predicting a prediction unit that ispredicted from among the plurality of prediction units, based onprediction values of at least one adjacent prediction unit among theplurality of prediction units.

The performing prediction may include generating residual values of theplurality of coding units by inter predicting all prediction unitsincluded in the plurality of coding units.

According to another aspect of an exemplary embodiment, there isprovided an apparatus for encoding an image, the apparatus including: apredictor that performs prediction on a plurality of coding units of theimage and generates a plurality of prediction units based on thepredicted plurality of coding units; a transformer that groups theplurality of prediction units into a transform unit and transformsresidual values included in the grouped plurality of prediction unitsinto a frequency domain, based on the transform unit, into frequencycomponent coefficients of the frequency domain; a quantizer thatquantizes the frequency component coefficients; and an entropy encoderthat entropy-encodes the quantized frequency component coefficients.

According to another aspect of an exemplary embodiment, there isprovided a method of decoding an image, the method including:entropy-decoding frequency component coefficients of a frequency domaingenerated from transformed residual values of a plurality of predictionunits of a transform unit, the plurality of prediction units included ina plurality of coding units of the image; inverse-quantizing theentropy-decoded frequency component coefficients; inverse-transformingthe inverse-quantized frequency component coefficients into a pixeldomain as restored residual values of the plurality of coding unitsincluded in the transform unit; and restoring the plurality of codingunits based on the restored residual values.

According to another aspect of an exemplary embodiment, there isprovided an apparatus for decoding an image, the apparatus including: anentropy decoder that entropy-encodes frequency component coefficients ofa frequency domain generated from transformed residual values ofplurality of prediction units of a transform unit, the plurality ofprediction units included in a plurality of coding units of the image;an inverse quantizer that inverse-quantizes the entropy-decodedfrequency component coefficients; an inverse transformer thatinverse-transforms the inverse-quantized frequency componentcoefficients into a pixel domain as restored residual values of theplurality of coding units included in the transform unit; and a restorerthat restores the plurality of coding units based on the restoredresidual values.

According to another aspect of an exemplary embodiment, there isprovided a computer readable recording medium having recorded thereon aprogram for executing the method of decoding and the method of encoding.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and/or other aspects will become more apparent by describingcertain exemplary embodiments, with reference to the accompanyingdrawings, in which:

FIG. 1 is a block diagram of an apparatus for encoding an image,according to an exemplary embodiment;

FIG. 2 is a block diagram of an apparatus for decoding an image,according to an exemplary embodiment;

FIG. 3 illustrates hierarchical coding units according to an exemplaryembodiment;

FIG. 4 is a block diagram of an image encoder based on a coding unit,according to an exemplary embodiment;

FIG. 5 is a block diagram of an image decoder based on a coding unit,according to an exemplary embodiment;

FIG. 6 illustrates a maximum coding unit, a sub coding unit, and aprediction unit, according to an exemplary embodiment;

FIG. 7 illustrates a coding unit and a transform unit, according to anexemplary embodiment;

FIGS. 8A, 8B, 8C, and 8D illustrate division shapes of a coding unit, aprediction unit, and a transform unit, according to an exemplaryembodiment;

FIG. 9 is a block diagram of an apparatus for encoding an image,according to another exemplary embodiment;

FIG. 10 is a diagram for describing a prediction method, according to anexemplary embodiment;

FIG. 11 is a block diagram of a transformer, according to an exemplaryembodiment;

FIGS. 12A through 12C are diagrams of types of transform units,according to exemplary embodiments;

FIGS. 13A through 13D are diagrams of types of transform units,according to other exemplary embodiments;

FIG. 14 is a diagram of different transform units, according toexemplary embodiments;

FIG. 15 is a block diagram of an apparatus for decoding an image,according to another exemplary embodiment;

FIG. 16 is a flowchart illustrating a method of encoding an image,according to an exemplary embodiment; and

FIG. 17 is a flowchart illustrating a method of decoding an image,according to an exemplary embodiment.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Certain exemplary embodiments are described in greater detail below withreference to the accompanying drawings. Expressions such as “at leastone of,” when preceding a list of elements, modify the entire list ofelements and do not modify the individual elements of the list. In thepresent specification, an “image” may denote a still image for a videoor a moving image, that is, the video itself.

In the following description, like drawing reference numerals are usedfor the like elements, even in different drawings. The matters definedin the description, such as detailed construction and elements, areprovided to assist in a comprehensive understanding of exemplaryembodiments. However, exemplary embodiments can be practiced withoutthose specifically defined matters.

FIG. 1 is a block diagram of an image encoding apparatus 100 forencoding an image, according to an exemplary embodiment. The imageencoding apparatus 100 may be implemented as a hardware apparatus suchas, for example, a processor of a computer or a computer system. Theimage encoding apparatus 100 may be also implemented as a softwaremodule residing on the computer system.

Referring to FIG. 1, the image encoding apparatus 100 includes a maximumencoding unit divider 110, an encoding depth determiner 120, an imagedata encoder 130, and an encoding information encoder 140 which may beimplemented, for example, as hardware or software modules integratedwithin the image encoding apparatus 100 or separately from the imageencoding apparatus 100.

The maximum encoding unit divider 110 may divide a current frame orslice based on a maximum coding unit that is a coding unit of thelargest size. That is, the maximum encoding unit divider 110 may dividethe current frame or slice into at least one maximum coding unit.

According to an exemplary embodiment, a coding unit may be representedusing a maximum coding unit and a depth. As described above, the maximumcoding unit indicates a coding unit having the largest size from amongcoding units of the current frame, and the depth indicates a degree ofhierarchically decreasing the coding unit. As a depth increases, acoding unit may decrease from a maximum coding unit to a minimum codingunit, wherein a depth of the maximum coding unit is defined as a minimumdepth and a depth of the minimum coding unit is defined as a maximumdepth. Since the size of a coding unit decreases from a maximum codingunit as a depth increases, a sub coding unit of a kth depth may includea plurality of sub coding units of a (k+n)th depth (k and n are integersequal to or greater than 1).

According to an increase of the size of a frame to be encoded, encodingan image in a greater coding unit may cause a higher image compressionratio. However, if a greater coding unit is fixed, an image may not beefficiently encoded by reflecting continuously changing imagecharacteristics.

For example, when a smooth area such as the sea or sky is encoded, thegreater a coding unit is, the more a compression ratio may increase.However, when a complex area such as people or buildings is encoded, thesmaller a coding unit is, the more a compression ratio may increase.

Accordingly, in an exemplary embodiment, a different maximum imagecoding unit and a different maximum depth are set for each frame orslice. Since a maximum depth denotes the maximum number of times bywhich a coding unit may decrease, the size of each minimum coding unitincluded in a maximum image coding unit may be variably set according toa maximum depth. The maximum depth may be determined differently foreach frame or slice or for each maximum coding unit.

The encoding depth determiner 120 determines a division shape of themaximum coding unit. The division shape may be determined based oncalculation of rate-distortion (RD) costs. The determined division shapeof the maximum coding unit is provided to the encoding informationencoder 140, and image data according to maximum coding units isprovided to the image data encoder 130.

A maximum coding unit may be divided into sub coding units havingdifferent sizes according to different depths, and the sub coding unitshaving different sizes, which are included in the maximum coding unit,may be predicted or frequency-transformed based on processing unitshaving different sizes. In other words, the image encoding apparatus 100may perform a plurality of processing operations for image encodingbased on processing units having various sizes and various shapes. Toencode image data, processing operations such as prediction,transformation, and entropy encoding are performed, wherein processingunits having the same size or different sizes may be used for everyoperation.

For example, the image encoding apparatus 100 may select a processingunit that is different from a coding unit to predict the coding unit.

When the size of a coding unit is 2N×2N (where N is a positive integer),processing units for prediction may be 2N×2N, 2N×N, N×2N, and N×N. Inother words, motion prediction may be performed based on a processingunit having a shape, whereby at least one of a height and a width of acoding unit is equally divided by two. Hereinafter, a processing unit,which is the base of prediction, is defined as a prediction unit.

A prediction mode may be at least one of an intra mode, an inter mode,and a skip mode, and a specific prediction mode may be performed foronly a prediction unit having a specific size or a specific shape. Forexample, the intra mode may be performed for only prediction unitshaving the sizes of 2N×2N or N×N and the shape of a square. Further, theskip mode may be performed for only a prediction unit having the size of2N×2N. If a plurality of prediction units exist in a coding unit, theprediction mode with the fewest encoding errors may be selected afterperforming prediction for every prediction unit.

Alternatively, the image encoding apparatus 100 may perform frequencytransform on image data based on a processing unit having a sizedifferent from a size of the coding unit. For the frequency transform inthe coding unit, the frequency transform may be performed based on aprocessing unit having a size equal to or smaller than that of thecoding unit. Hereinafter, a processing unit, which is the base offrequency transform, is defined as a transform unit. The frequencytransform may be discrete cosine transform (DCT) or Karhunen Loevetransform (KLT).

The encoding depth determiner 120 may determine sub coding unitsincluded in a maximum coding unit using RD optimization based on aLagrangian multiplier. In other words, the encoding depth determiner 120may determine a shape of a plurality of sub coding units divided fromthe maximum coding unit, wherein the sub coding units have differentsizes according to the depths of sub coding units. The image dataencoder 130 outputs a bitstream by encoding the maximum coding unitbased on the division shapes determined by the encoding depth determiner120.

The encoding information encoder 140 encodes information about anencoding mode of the maximum coding unit determined by the encodingdepth determiner 120. In other words, the encoding information encoder140 outputs a bitstream by encoding information about a division shapeof the maximum coding unit, information about the maximum depth, andinformation about an encoding mode of a sub coding unit for each depth.The information about the encoding mode of the sub coding unit mayinclude information about a prediction unit of the sub coding unit,information about a prediction mode for each prediction unit, andinformation about a transform unit of the sub coding unit.

The information about the division shape of the maximum coding unit maybe flag information, indicating whether each coding unit is divided. Forexample, when the maximum coding unit is divided and encoded,information indicating whether the maximum coding unit is divided isencoded. Also, when a sub coding unit divided from the maximum codingunit is divided and encoded, information indicating whether the subcoding unit is divided is encoded.

Since sub coding units having different sizes exist for each maximumcoding unit and information about an encoding mode is determined foreach sub coding unit, information about at least one encoding mode maybe determined for one maximum coding unit.

The image encoding apparatus 100 may generate sub coding units byequally dividing the height and width of a maximum coding unit by twoaccording to an increase of depth. That is, when the size of a codingunit of a kth depth is 2N×2N, the size of a coding unit of a (k+1)thdepth is N×N.

Accordingly, the image encoding apparatus 100 may determine an optimaldivision shape for each maximum coding unit based on sizes of maximumcoding units and a maximum depth in consideration of imagecharacteristics. By variably adjusting the size of a maximum coding unitin consideration of image characteristics and encoding an image throughdivision of a maximum coding unit into sub coding units of differentdepths, images having various resolutions may be more efficientlyencoded.

FIG. 2 is a block diagram of an image decoding apparatus 200 fordecoding an image according to an exemplary embodiment. The imagedecoding apparatus 200 may be implemented as a hardware apparatus suchas, for example, a processor of a computer, or a computer system. Theimage decoding apparatus 200 may be also implemented as a softwaremodule residing on the computer system.

Referring to FIG. 2, the image decoding apparatus 200 includes an imagedata acquisition unit 210, an encoding information extractor 220, and animage data decoder 230 which may be implemented, for example, ashardware or software modules integrated within the image decodingapparatus 200 or separately from the image encoding apparatus 200.

The image data acquisition unit 210 acquires image data according tomaximum coding units by parsing a bitstream received by the imagedecoding apparatus 200 and outputs the image data to the image datadecoder 230. The image data acquisition unit 210 may extract informationabout a maximum coding unit of a current frame or slice from a header ofthe current frame or slice. In other words, the image data acquisitionunit 210 divides the bitstream in the maximum coding unit so that theimage data decoder 230 may decode the image data according to maximumcoding units.

The encoding information extractor 220 extracts information about amaximum coding unit, a maximum depth, a division shape of the maximumcoding unit, an encoding mode of sub coding units from the header of thecurrent frame by parsing the bitstream received by the image decodingapparatus 200. The information about a division shape and theinformation about an encoding mode are provided to the image datadecoder 230.

The information about a division shape of the maximum coding unit mayinclude information about sub coding units having different sizesaccording to depths and included in the maximum coding unit, and may beflag information indicating whether each coding unit is divided.

The information about an encoding mode may include information about aprediction unit according to sub coding units, information about aprediction mode, and information about a transform unit.

The image data decoder 230 restores the current frame by decoding imagedata of every maximum coding unit based on the information extracted bythe encoding information extractor 220.

The image data decoder 230 may decode sub coding units included in amaximum coding unit based on the information about a division shape ofthe maximum coding unit. A decoding process may include a predictionprocess including intra prediction and motion compensation and aninverse transform process.

The image data decoder 230 may perform intra prediction or interprediction based on information about a prediction unit and informationabout a prediction mode to predict a prediction unit. The image datadecoder 230 may also perform inverse transform for each sub coding unitbased on information about a transform unit of a sub coding unit.

FIG. 3 illustrates hierarchical coding units according to an exemplaryembodiment.

Referring to FIG. 3, the hierarchical coding units may include codingunits whose widths and heights are 64×64, 32×32, 16×16, 8×8, and 4×4.Besides these coding units having perfect square shapes, coding unitswhose widths and heights are 64×32, 32×64, 32×16, 16×32, 16×8, 8×16,8×4, and 4×8 may also exist.

Referring to FIG. 3, for image data set 310 whose resolution is1920×1080, the size of a maximum coding unit is set to 64×64, and amaximum depth is set to 2.

For image data set 320 whose resolution is 1920×1080, the size of amaximum coding unit is set to 64×64, and a maximum depth is set to 3.For image data set 330 whose resolution is 352×288, the size of amaximum coding unit is set to 16×16, and a maximum depth is set to 1.

When the resolution is high or the amount of data is great, a maximumsize of a coding unit may be set relatively great to increase acompression ratio and reflect image characteristics more precisely.Accordingly, for the image data sets 310 and 320 having higherresolution than the image data set 330, 64×64 may be selected as thesize of a maximum coding unit.

A maximum depth indicates the total number of layers in the hierarchicalcoding units. Since the maximum depth of the image data set 310 is 2, acoding unit 315 of the image data set 310 may include a maximum codingunit whose longer axis size is 64 and sub coding units whose longer axissizes are 32 and 16, according to an increase of a depth.

On the other hand, since the maximum depth of the image data set 330 is1, a coding unit 335 of the image data set 330 may include a maximumcoding unit whose longer axis size is 16 and coding units whose longeraxis sizes is 8, according to an increase of a depth.

However, since the maximum depth of the image data 320 is 3, a codingunit 325 of the image data set 320 may include a maximum coding unitwhose longer axis size is 64 and sub coding units whose longer axissizes are 32, 16, 8 and 4 according to an increase of a depth. Since animage is encoded based on a smaller sub coding unit as a depthincreases, exemplary embodiments are suitable for encoding an imageincluding more minute scenes.

FIG. 4 is a block diagram of an image encoder 400 based on a codingunit, according to an exemplary embodiment. The image encoder 400 may beimplemented as a hardware device such as, for example, a processor of acomputer or as a software module residing on the computer system.

An intra predictor 410 performs intra prediction on prediction units ofthe intra mode in a current frame 405, and a motion estimator 420 and amotion compensator 425 perform inter prediction and motion compensationon prediction units of the inter mode using the current frame 405 and areference frame 495. The intra predictor 410, the motion estimator 420,the motion compensator 425, and the reference frame 495 may beimplemented, for example, as hardware or software modules integratedwithin the image encoder 400 or separately from the image encoder 400.

Residual values are generated based on the prediction units output fromthe intra predictor 410, the motion estimator 420, and the motioncompensator 425. The generated residual values are output as quantizedtransform coefficients by passing through a transformer 430 and aquantizer 440.

The quantized transform coefficients are restored to residual values bypassing through an inverse quantizer 460 and an inverse transformer 470,and the restored residual values are post-processed by passing through adeblocking unit 480 and a loop filtering unit 490 and output as thereference frame 495. The quantized transform coefficients may be outputas a bitstream 455 by passing through an entropy encoder 450.

To perform encoding based on an encoding method according to anexemplary embodiment, the intra predictor 410, the motion estimator 420,the motion compensator 425, the transformer 430, the quantizer 440, theentropy encoder 450, the inverse quantizer 460, the inverse transformer470, the deblocking unit 480, and the loop filtering unit 490 of theimage encoder 400 perform image encoding processes based on a maximumcoding unit, a sub coding unit according to depths, a prediction unit,and a transform unit.

FIG. 5 is a block diagram of an image decoder 500 based on a codingunit, according to an exemplary embodiment. The image decoder 500 may beimplemented as a hardware device such as, for example, a processor of acomputer or as a software module residing on the computer system.

A bitstream 505 passes through a parser 510 so that the encoded imagedata to be decoded and encoding information necessary for decoding areparsed. The encoded image data is output as inverse-quantized data bypassing through an entropy decoder 520 and an inverse quantizer 530 andrestored to residual values by passing through an inverse transformer540. The residual values are restored according to coding units by beingadded to an intra prediction result of an intra predictor 550 or amotion compensation result of a motion compensator 560. The restoredcoding units 585, 595 are used for prediction of next coding units or anext frame by passing through a deblocking unit 570 and a loop filteringunit 580. The parser 510, the entropy decoder 520, the inverse quantizer530, the inverse transformer 540, the intra predictor 550, thecompensator 560, the deblocking unit 570, and the loop filtering unit580 may be implemented, for example, as hardware or software modulesintegrated within the image decoder 500 or separately from the imagedecoder 500.

To perform decoding based on a decoding method according to an exemplaryembodiment, the parser 510, the entropy decoder 520, the inversequantizer 530, the inverse transformer 540, the intra predictor 550, themotion compensator 560, the deblocking unit 570, and the loop filteringunit 580 of the image decoder 500 perform image decoding processes basedon a maximum coding unit, a sub coding unit according to depths, aprediction unit, and a transform unit.

In particular, the intra predictor 550 and the motion compensator 560determine a prediction unit and a prediction mode in a sub coding unitby considering a maximum coding unit and a depth, and the inversetransformer 540 performs inverse transform by considering the size of atransform unit.

FIG. 6 illustrates a maximum coding unit, a sub coding unit, and aprediction unit, according to an exemplary embodiment.

The image encoding apparatus 100 illustrated in FIG. 1 and the imagedecoding apparatus 200 illustrated in FIG. 2 use hierarchical codingunits to perform encoding and decoding in consideration of imagecharacteristics. A maximum coding unit and a maximum depth may beadaptively set according to the image characteristics or variously setaccording to requirements of a user.

In FIG. 6, a hierarchical coding unit structure 600 has a maximumencoding unit 610 which is a maximum coding unit whose height and widthare 64 and maximum depth is 4. A depth increases along a vertical axisof the hierarchical coding unit structure 600, and as a depth increases,heights and widths of sub coding units 620 to 650 decrease. Predictionunits of the maximum encoding unit 610 and the sub coding units 620 to650 are shown along a horizontal axis of the hierarchical coding unitstructure 600.

The maximum encoding unit 610 has a depth of 0 and the size of an codingunit, or a height and a width, of 64×64. A depth increases along thevertical axis, and there exist a first sub coding unit 620 whose size is32×32 and depth is 1, a second sub coding unit 630 whose size is 16×16and depth is 2, a third sub coding unit 640 whose size is 8×8 and depthis 3, and a minimum encoding unit 650 whose size is 4×4 and depth is 4.The minimum encoding unit 650 whose size is 4×4 and depth is 4 is aminimum coding unit, and the minimum coding unit may be divided intoprediction units, each of which is a size smaller than the minimumcoding unit.

Referring to FIG. 6, examples of prediction units are shown along thehorizontal axis according to each depth. That is, a prediction unit ofthe maximum encoding unit 610 whose depth is 0 may be a prediction unitwhose size is equal to the size 64×64 of the maximum coding unit, or aprediction unit 612 whose size is 64×32, a prediction unit 614 whosesize is 32×64, or a prediction unit 616 whose size is 32×32, which has asize smaller than that of the maximum coding unit whose size is 64×64.

A prediction unit of the first sub coding unit 620 whose depth is 1 andsize is 32×32 may be a prediction unit whose size is equal to the size32×32 of the first sub coding unit, or a prediction unit 622 whose sizeis 32×16, a prediction unit 624 whose size is 16×32, or a predictionunit 626 whose size is 16×16, which has a size smaller than that of thefirst sub coding unit 620 whose size is 32×32.

A prediction unit of the second sub coding unit 630 whose depth is 2 andsize is 16×16 may be a prediction unit whose size is equal to the size16×16 of the second sub coding unit 630, or a prediction unit 632 whosesize is 16×8, a prediction unit 634 whose size is 8×16, or a predictionunit 636 whose size is 8×8, which has a size smaller than that of thesecond sub coding unit 630 whose size is 16×16.

A prediction unit of the third sub coding unit 640 whose depth is 3 andsize is 8×8 may be a prediction unit whose size is equal to the size 8×8of the third sub coding unit 640 or a prediction unit 642 whose size is8×4, a prediction unit 644 whose size is 4×8, or a prediction unit 646whose size is 4×4, which has a size smaller than that of the third subcoding unit 640 whose size is 8×8.

The minimum encoding unit 650 whose depth is 4 and size is 4×4 is aminimum coding unit and a coding unit of a maximum depth. A predictionunit of the minimum encoding unit 650 may be a prediction unit 650 whosesize is 4×4, a prediction unit 652 having a size of 4×2, a predictionunit 654 having a size of 2×4, or a prediction unit 656 having a size of2×2.

FIG. 7 illustrates a coding unit and a transform unit, according to anexemplary embodiment.

The image encoding apparatus 100 illustrated in FIG. 1 and the imagedecoding apparatus 200 illustrated in FIG. 2 perform encoding anddecoding with a maximum coding unit or with sub coding units, which havesize equal to or smaller than the maximum coding unit, divided from themaximum coding unit. In the encoding and decoding process, the size of atransform unit for frequency transform is selected to be no larger thanthat of a corresponding coding unit. For example, if a current codingunit 710 has the size of 64×64, frequency transform may be performedusing a transform unit 720 having the size of 32×32.

FIGS. 8A, 8B, 8C, and 8D illustrate division shapes of a coding unit, aprediction unit, and a transform unit, according to an exemplaryembodiment.

FIGS. 8A and 8B respectively illustrate a coding unit and a predictionunit, according to an exemplary embodiment.

FIG. 8A shows a division shape selected by the image encoding apparatus100 illustrated in FIG. 1, to encode a maximum coding unit 810. Theimage encoding apparatus 100 divides the maximum coding unit 810 intovarious shapes, performs encoding, and selects an optimal division shapeby comparing encoding results of various division shapes with each otherbased on the RD costs. When it is optimal that the maximum coding unit810 to be encoded, the maximum coding unit 810 may be encoded withoutdividing the maximum coding unit 810, as illustrated in FIGS. 8A through8D.

Referring to FIG. 8A, the maximum coding unit 810 whose depth is 0 isencoded by dividing the maximum encoding unit 810 into sub coding units812, 854 whose depths are equal to or greater than 1. That is, themaximum coding unit 810 is divided into 4 sub coding units whose depthsare 1, and all or some of the sub coding units whose depths are 1 aredivided into sub coding units 814, 816, 818, 828, 850, and 852 whosedepths are 2.

A sub coding unit located in an upper-right side and a sub coding unitlocated in a lower-left side among the sub coding units whose depths are1 are divided into sub coding units whose depths are equal to or greaterthan 2. Some of the sub coding units whose depths are equal to orgreater than 2 may be further divided into sub coding units 820, 822,824, 826, 830, 832, 840, 842, 844, 846, and 848 whose depths are equalto or greater than 3.

FIG. 8B shows a division shape of a prediction unit for the maximumcoding unit 810.

Referring to FIG. 8B, a prediction unit 860 for the maximum coding unit810 may be divided differently from the maximum coding unit 810. Inother words, a prediction unit for each of sub coding units may besmaller than a corresponding sub coding unit.

For example, a prediction unit for a sub coding unit 854 located in alower-right side among the sub coding units 812, 854 whose depths are 1may be smaller than the sub coding unit 854. In addition, predictionunits for sub coding units 814, 816, 850, and 852 of sub coding units814, 816, 818, 828, 850, and 852 whose depths are 2 may be smaller thanthe sub coding units 814, 816, 850, and 852, respectively.

In addition, prediction units for sub coding units 822, 832, and 848whose depths are 3 may be smaller than the sub coding units 822, 832,and 848, respectively. The prediction units may have a shape wherebyrespective sub coding units are equally divided by two in a direction ofheight or width or have a shape whereby respective sub coding units areequally divided by four in directions of height and width.

FIGS. 8C and 8D illustrate a prediction unit and a transform unit,according to an exemplary embodiment.

FIG. 8C shows a division shape of a prediction unit for the maximumcoding unit 810 shown in FIG. 8B, and FIG. 8D shows a division shape ofa transform unit of the maximum coding unit 810.

Referring to FIG. 8D, a division shape of a transform unit 870 may beset differently from the prediction unit 860.

For example, even though a prediction unit for the sub coding unit 854whose depth is 1 is selected with a shape whereby the height of the subcoding unit 854 is equally divided by two, a transform unit may beselected with the original size of the sub coding unit 854. Likewise,even though prediction units for sub coding units 814 and 850 whosedepths are 2 are selected with a shape whereby the height of each of thesub coding units 814 and 850 is equally divided by two, a transform unitmay be selected with the same size as the original size of each of thesub coding units 814 and 850.

A transform unit may be selected with a smaller size than a predictionunit. For example, when a prediction unit for the sub coding unit 852whose depth is 2 is selected with a shape whereby the width of the subcoding unit 852 is equally divided by two, a transform unit may beselected with a shape whereby the sub coding unit 852 is equally dividedby four in directions of height and width, which has a smaller size thanthe shape of the prediction unit.

Alternatively, as will described with reference to FIGS. 13A through13D, a transform unit may be set to have a larger size than a codingunit, regardless of the coding unit.

FIG. 9 is a block diagram of an apparatus 900 for encoding an image,according to another exemplary embodiment.

Referring to FIG. 9, the image encoding apparatus 900 according to thecurrent exemplary embodiment includes a predictor 910, a transformer920, a quantizer 930, and an entropy encoder 940.

The predictor 910 generates residual values by performing intraprediction or inter prediction on one or more coding units. As will bedescribed later, residual values included in a plurality of predictionunits may be grouped into one transform unit and then transformed to afrequency domain, and thus the residual values are generated bypredicting the one or more coding units based on the plurality ofprediction units. The transform to the frequency domain may be DCT orKLT.

As described above with reference to FIG. 8A, in the image encodingmethod according to an exemplary embodiment, one coding unit may includea plurality of prediction units. Thus, the predictor 910 may predicteach of the prediction units, and generate the residual values of theprediction units included in the one coding unit.

Alternatively, the prediction unit 910 may predict the plurality ofcoding units all at once. As will be described later, according to anexemplary embodiment, a plurality of prediction units included in aplurality of coding units may be grouped into one transform unit, andthus residual values are generated by predicting each of the predictionunits included in the coding units. For example, all sub coding unitsincluded in one maximum coding unit may be predicted in order togenerate the residual values of the coding units.

According to conventional technology, since transform (e.g. DCT or KLT)is performed with a size smaller than or equal to a prediction unit, apredetermined prediction unit is independently encoded, restored, andthen used to predict a next prediction unit. However, according to amethod of encoding an image, according to an exemplary embodiment, whichwill be described later, since transform is performed by groupingprediction units included in one or more coding units into one transformunit, a predetermined prediction unit cannot be independently encodedand restored. This will be described in detail with reference to FIG.10.

FIG. 10 is a diagram for describing a prediction method, according to anexemplary embodiment.

Referring to FIG. 10, one coding unit 1000 may include a plurality ofprediction units 1010 through 1040. If transform is performed with asize smaller than or equal to a prediction unit, as in conventionaltechnology, the prediction units 1010 through 1030 may be encoded andrestored before encoding the prediction unit 1040 at a lower-right side.

Accordingly, if the prediction unit 1040 is to be predicted via intraprediction according to the conventional technology, the prediction unit1040 is intra predicted by using pixels adjacent to the prediction unit1040, from among pixels generated by encoding and then restoring theprediction units 1010 through 1030.

On the other hand, according to an exemplary embodiment, a plurality ofprediction units are grouped into one transform unit, and then transformis performed. Here, if the prediction units 1010 through 1040 of FIG. 10are grouped into one transform unit, the prediction unit 1040 at thelower-right side is encoded with the other prediction units 1010 through1030, and thus the prediction units 1010 through 1030 are not encodedbefore encoding the prediction unit 1040. Accordingly, the predictionunit 1040 cannot be intra predicted by using the pixels generated byencoding and then restoring the prediction units 1010 through 1030.

Consequently, the prediction unit 910 of FIG. 9 may predict theprediction unit 1040 by using prediction values of the prediction units1010 through 1030. The prediction unit 1040 at the lower-right side ispredicted by using the prediction values of the prediction units 1010through 1030, instead of the pixels generated by encoding and thenrestoring the prediction units 1010 through 1030.

In other words, if there is a first prediction unit predicted via intraprediction, from among prediction units grouped into one transform unit,the first prediction unit may be intra predicted by using predictionvalues of at least one adjacent prediction unit.

Alternatively, the prediction units grouped into one transform unit mayall be predicted via inter prediction. As described with reference toFIG. 10, since a prediction unit that is predicted via intra predictionis at issue while grouping a plurality of prediction units into onetransform unit, all prediction units grouped into the transform unit maybe predicted by using only inter prediction.

Referring back to FIG. 9, the transformer 920 receives an imageprocessing unit in a pixel domain, and transforms the image processingunit into a frequency domain. The transformer 920 transforms theresidual values generated by the prediction unit 910 into the frequencydomain.

As described above, the transformer 920 groups the prediction units intoone transform unit, and performs DCT or KLT according to the transformunit. The residual values may be residual values of a plurality ofprediction units included in one or more coding units. Coefficients offrequency components are generated as a result of transforming the pixeldomain to the frequency domain.

According to an exemplary embodiment, the transform to the frequencydomain may be performed via DCT or KLT, and discrete cosine coefficientsare generated as a result of the DCT or KLT. However, any transform fortransforming an image in a pixel domain to the frequency domain may beused.

FIG. 11 is a block diagram of the transformer 920 according to anexemplary embodiment.

Referring to FIG. 11, the transformer 920 includes a selector 1110 and atransform performer 1120.

The selector 1110 sets one transform unit by selecting a plurality ofadjacent prediction units. According to conventional image encodingapparatuses described above, intra prediction or inter prediction isperformed according to a predetermined prediction unit and DCT or KLT isperformed with a size smaller than or equal to the predeterminedprediction unit. In other words, the conventional image encodingapparatuses perform DCT or KLT based on a transform unit having a sizesmaller than or equal to a prediction unit.

However, a compression ratio of image encoding is deteriorated since anadded overhead increases as a size of a transform unit is decreased dueto header information added for each transform unit. Accordingly, theimage encoding apparatus 900 according to the current exemplaryembodiment groups the adjacent prediction units into one transform unit,and then performs DCT or KLT according to the transform unit.Specifically, since it is highly likely that the adjacent predictionunits have similar residual values, a compression ratio of encoding maybe remarkably increased when DCT or KLT is performed according to thetransform unit generated by grouping the adjacent prediction units.

Accordingly, the selector 1110 selects the prediction units to begrouped into one transform unit and on which DCT or KLT is to beperformed. The prediction units may be adjacent to each other. This willbe described in detail with reference to FIGS. 12A through 12C and 13Athrough 13D.

FIGS. 12A through 12C are diagrams of types of transform units 1230through 1250, according to exemplary embodiments.

Referring to FIGS. 12A through 12C, a prediction unit 1220 may have ashape whereby a coding unit 1210 is equally divided by two in adirection of width. The coding unit 1210 may be a maximum coding unit asdescribed above, or a sub coding unit having a smaller size than themaximum coding unit.

Even when the coding unit 1210 and the prediction unit 1220 areidentical, the transform units 1230 through 1250 may be different. Asize of the transform unit 1230 may be smaller than that of theprediction unit 1220 as shown in FIG. 12A, or a size of the transformunit 1240 may be identical to that of the prediction unit 1220 as shownin FIG. 12B. Alternatively, a size of the transform unit 1250 may belarger than that of the prediction unit 1220 as shown in FIG. 12C.

The prediction units grouped into one transform unit may be a pluralityof prediction units included in one coding unit as shown in FIGS. 12Athrough 12C, or may be a plurality of prediction units included indifferent coding units. In other words, a plurality of prediction unitsincluded in at least one coding unit may be grouped into one transformunit and then transformed to the frequency domain.

FIGS. 13A through 13D are diagrams of types of transform units accordingto exemplary embodiments.

One maximum coding unit 1300 may be divided into sub coding units 1302through 1308 having different sizes and then encoded as shown in FIG.13A, and each of the sub coding units 1302 through 1308 may include atleast one prediction unit 1310 through 1340, as shown in FIG. 13B.

The selector 1110 may group the prediction units 1310 through 1340 shownin FIG. 13B into one transform unit 1350 shown in FIG. 13C, and thentransform the transform unit 1350 into the frequency domain.

Alternatively, the selector 1110 may group the prediction units 1310 and1330 through 1339 of the sub coding units 1302 and 1306 on the left intoone transform unit 1360, and group the prediction units 1320 through1328 and 1340 of the sub coding units 1304 and 1308 on the right intoone transform unit 1362, as shown in FIG. 13D.

Referring back to FIG. 11, a criterion for the selector 1110 to select aplurality of adjacent prediction units is not limited. However,according to an exemplary embodiment, the selector 1110 may select atransform unit based on a depth. As described above, the depth indicatesa degree of hierarchically decreasing a coding unit from a maximumcoding unit of a current slice or frame to sub coding units. Asdescribed above with reference to FIGS. 3 and 6, as a depth increases, asize of a sub coding unit decreases, and thus a size of a predictionunit included in the sub coding unit decreases. Here, when DCT or KLT isperformed according to a transform unit having a size smaller than orequal to a prediction unit, a compression ratio of image encoding isdecreased because header information is added for each transform unit asdescribed above.

Accordingly, prediction units included in a sub coding unit whose depthis equal to or above a predetermined value may be grouped into onetransform unit, and then DCT or KLT may be performed on the transformunit. Thus, the selector 1110 may set the transform unit based on thedepth of the sub coding unit. For example, when a depth of the codingunit 1210 of FIG. 12C is higher than k, the selector 1110 groups theprediction units 1220 into one transform unit 1250.

Alternatively, when a maximum coding unit includes a plurality of subcoding units whose depths are equal to or above a predetermined value,the selector 1110 may group prediction units of the sub coding unitsinto one transform unit. FIG. 13C illustrates an example of groupingprediction units of sub coding units whose depth is larger than amaximum coding unit, i.e., whose depth is larger than 1, into onetransform unit.

According to another exemplary embodiment, the selector 1110 may set aplurality of adjacent prediction units, on which prediction is performedaccording to a same type of prediction mode, into one transform unit.The adjacent prediction units that are predicted by using intraprediction or inter prediction are grouped into one transform unit.Since it is highly likely that the adjacent prediction units that arepredicted according to the same type of prediction mode have similarresidual values, DCT or KLT may be performed by grouping the adjacentprediction units into one transform unit.

When the selector 1110 sets the transform unit, the transform performer1120 transforms the adjacent prediction units into a frequency domainaccording to the set transform unit. Coefficients of frequency domain(e.g. discrete cosine coefficients) are generated by transforming theselected prediction units into one transform unit.

Referring back to FIG. 9, the quantizer 930 quantizes the frequencycomponent coefficients generated by the transformer 920. The quantizer930 may quantize the coefficients input according to a predeterminedquantization process.

The entropy encoder 940 entropy-encodes the coefficients quantized bythe quantizer 930. Here, the discrete cosine coefficients may beentropy-encoded by using context-adaptive binary arithmetic coding(CABAC) or context-adaptive variable length coding (CAVLC).

The image encoding apparatus 900 may encode flag information indicatingwhether the transform unit generated by grouping the prediction unitsincludes the coefficients. If there are no coefficients to beentropy-encoded, i.e., when the quantized coefficients are all ‘0’, flaginformation indicating that the transform unit does not include thecoefficients is encoded, and the quantized coefficients are notseparately entropy-encoded.

The image encoding apparatus 900 according to the current exemplaryembodiment may determine an optimum transform unit by repeatedlyperforming transform, quantization, and entropy encoding on differenttransform units. The optimum transform unit may be determined bymechanically repeating a process of selecting a plurality of predictionunits by using various methods, instead of selecting the predictionunits based on a predetermined criterion, such as a depth or a same typeof prediction mode. The optimum transform unit may be determined basedon calculation of RD costs, and this will be described in detail withreference to FIG. 14.

FIG. 14 is a diagram of different transform units 1430 through 1460according to exemplary embodiments.

Referring to FIG. 14, the image encoding apparatus 900 repeatedlyencodes different transform units 1430 through 1460.

As shown in FIG. 14, a coding unit 1410 may be predicted and encodedbased on a prediction unit 1420 having a size smaller than the codingunit 1410. DCT or KLT is performed on residual values generated as aresult of prediction, and here, the DCT or KLT may be performed based onthe different transform units 1430 through 1460 as shown in FIG. 14.

The transform unit 1430 has the same size as the coding unit 1410, andis generated by grouping all prediction units included in the codingunit 1410.

The transform units 1440 have a size whereby the coding unit 1410 isequally divided by two in a direction of width, and are generated bygrouping the prediction units that are adjacent in a vertical direction.

The transform units 1450 have a size whereby the coding unit 1410 isequally divided by two in a direction of height, and are generated bygrouping the prediction units that are adjacent in a horizontaldirection.

The transform units 1460 have the same sizes as the prediction units1420.

The image encoding apparatus 900 may determine the optimum transformunit by repeatedly performing transform, quantization, and entropyencoding on the transform units 1430 through 1460.

Alternatively, the image encoding apparatus 900 may encode flaginformation indicating whether the transform unit is generated bygrouping a plurality of prediction units included in one or more codingunits. For example, when a transform unit is set by grouping a pluralityof prediction units included in one coding unit as shown in FIGS. 12Athrough 12C, flag information is set to ‘0’, and when a transform unitis set by grouping a plurality of prediction units included in aplurality of coding units as shown in FIGS. 13A through 13D, flaginformation is set to ‘1’.

FIG. 14 illustrates an example of determining the optimum transform unitwhen one transform unit is set by grouping prediction units included inone coding unit. However, the optimum transform unit may be determinedby repeatedly performing DCT, quantization, and entropy encoding ondifferent transform units, as shown in FIG. 14, even when one transformunit is set by grouping prediction units included in a plurality ofcoding units.

FIG. 15 is a block diagram of an apparatus 1500 for decoding an image,according to another exemplary embodiment.

Referring to FIG. 15, the image decoding apparatus 1500 includes anentropy decoder 1510, an inverse quantizer 1520, an inverse transformer1530, and a restorer 1540.

The entropy decoder 1510 entropy-decodes frequency componentcoefficients of a predetermined transform unit. As described above withreference to FIGS. 12A through 12C and 13A through 13D, the transformunit may be generated by grouping a plurality of prediction units. Asdescribed above, the prediction units may be adjacent to each other, andmay be included in one coding unit or in a plurality of different codingunits.

As described above with reference to the image encoding apparatus 900,the transform unit may be generated by grouping a plurality of adjacentprediction units based on a depth, or by grouping a plurality ofadjacent prediction units on which prediction is performed according toa same type of prediction mode, i.e., according to an intra predictionmode or an inter prediction mode. Alternatively, as described withreference to FIG. 14, an optimum transform unit may be selected byrepeatedly performing transform, quantization, and entropy decoding ondifferent transform units by mechanically repeating a process ofgrouping a plurality of prediction units.

If a transform unit does not include coefficients (e.g. discrete cosinecoefficients), the entropy decoder 1510 may not separatelyentropy-decode quantized coefficients. If the transform unit does notinclude the quantized coefficients, the quantized coefficients are notseparately entropy-encoded by referring to predetermined flaginformation.

The inverse quantizer 1520 inverse-quantizes the frequency componentcoefficients that are entropy-decoded by the entropy decoder 1510. Thefrequency component coefficients that are entropy-decoded according to aquantization step used while encoding the transform unit areinverse-quantized.

The inverse transformer 1530 inverse-transforms the inverse-quantizedfrequency component coefficients into a pixel domain. Inverse DCT orinverse KLT is performed on the inverse-quantized discrete cosinecoefficients to restore a transform unit in a pixel domain. As a resultof inverse transform, residual values of the transform unit arerestored.

The restored transform unit includes a plurality of prediction units,and as described above, the prediction units may be included in onecoding unit or in a plurality of different coding units.

The restorer 1540 generates prediction values by predicting a pluralityof prediction units included in the restored transform unit. Predictionvalues of one coding unit are generated if the prediction units groupedinto one transform unit are included in one coding unit, and predictionvalues of a plurality of coding units are generated if the predictionunits grouped into one transform unit are included in a plurality ofcoding units. One coding unit or a plurality of coding units is restoredby adding the generated prediction values and the residual valuesrestored by the inverse transformer 1530.

Whether the prediction values are generated for one coding unit or aplurality of coding units may be determined based on flag informationindicating whether the image encoding apparatus 900 generated atransform unit by grouping a plurality of prediction units included inone coding unit or in a plurality of coding units.

According to one exemplary embodiment, if the prediction units groupedinto one transform unit include a prediction unit that isintra-predicted, intra prediction may be performed based on predictionvalues of at least one adjacent prediction unit, as described withreference to FIG. 10. Alternatively, a plurality of prediction unitsgrouped into one transform unit may all be predicted by using interprediction.

FIG. 16 is a flowchart illustrating a method of encoding an image,according to an exemplary embodiment.

Referring to FIG. 16, an apparatus for encoding an image generatesresidual values by performing prediction on one or more coding units inoperation 1610.

A plurality of prediction units grouped into one transform unit may beincluded in one coding unit or in a plurality of coding units.Accordingly, when the prediction units are included in one coding unit,the residual values are generated by performing prediction on one codingunit, and when the prediction units are included in a plurality ofcoding units, the residual values are generated by performing predictionon the plurality of coding units.

A method of generating the residual values by predicting the predictionunits all at once has been described above with reference to FIG. 10.

In operation 1620, the apparatus sets one transform unit by selecting aplurality of prediction units. The prediction units may be included inone coding unit or in a plurality of coding units. The adjacentprediction units may be selected based on depth, or adjacent predictionunits on which prediction is performed in a same type of prediction modemay be selected.

In operation 1630, the apparatus transforms the prediction units into afrequency domain according to the transform unit set in operation 1620.Coefficients of frequency domain are generated by performing transformon the transform unit set by grouping the prediction units.

In operation 1640, the apparatus quantizes frequency componentcoefficients, e.g. the discrete cosine coefficients generated inoperation 1630, according to a predetermined quantization process.

In operation 1650, the apparatus entropy-encodes the frequency componentcoefficients quantized in operation 1640. The entropy encoding isperformed via CABAC or CAVLC.

As described with reference to FIG. 14, the method may further includesetting an optimum transform unit by repeating operations 1610 through1640 on different transform units. The optimum transform unit may be setby repeatedly performing transform, quantization, and entropy encodingon the different transform units as shown in FIG. 14.

FIG. 17 is a flowchart illustrating a method of decoding an image,according to an exemplary embodiment.

Referring to FIG. 17, the apparatus entropy-decodes frequency componentcoefficients of a predetermined transform unit, in operation 1710. Thefrequency component coefficients may be discrete cosine coefficients.The transform unit may be set by grouping a plurality of predictionunits. As described above, the prediction units may be adjacent to eachother, and may be included in one coding unit or in a plurality ofdifferent coding units.

In operation 1720, the apparatus inverse-quantizes the frequencycomponent coefficients that are entropy-decoded in operation 1710. Thediscrete cosine coefficients are inverse-quantized by using aquantization step used during encoding.

In operation 1730, the apparatus inverse-transforms the frequencycomponent coefficients that are inverse-quantized in operation 1720 intoa pixel domain to restore a transform unit. The restored transform unitis set by grouping a plurality of prediction units. Residual valuesincluded in the transform unit are restored. Residual values of onecoding unit are restored if the prediction units are included in onecoding unit, and residual values of a plurality of coding units arerestored if the prediction units are included in the coding units.

As described above, the transform unit may be set by grouping adjacentprediction units based on a depth, or by grouping adjacent predictionunits on which prediction is performed according to a same type ofprediction mode.

In operation 1740, the apparatus restores the one or more coding unitsbased on the residual values included in the transform unit restored inoperation 1730. Prediction values are generated by predicting the one ormore coding units, and the one or more coding units are restored byadding the generated prediction values and the residual values restoredin operation 1730. A method of predicting the prediction values includedin one or more coding units has been described above with reference toFIG. 10.

If the transform unit is set by grouping the prediction units includedin one coding unit, one coding unit is restored, and if the transformunit is set by grouping the prediction units included in a plurality ofcoding units, the plurality of coding units are restored.

According to the exemplary embodiments, an image is more efficientlycompressed and encoded since a transform unit can be set to have a sizelarger than a prediction unit, and transform can be performed on thetransform unit.

While the present invention has been particularly shown and describedwith reference to exemplary embodiments thereof, it will be understoodby one of ordinary skill in the art that various changes in form anddetails may be made therein without departing from the spirit and scopeof the invention as defined by the following claims and theirequivalents. Also, the exemplary embodiments can also be embodied ascomputer readable codes on a computer readable recording medium.

The image encoding or decoding apparatus or the image encoder or decoderillustrated in FIG. 1, 2, 4, 5, 9, 11, or 15 may include a bus coupledto every unit of the apparatus or encoder or decoder, at least oneprocessor that is connected to the bus and is for executing commands,and memory connected to the bus to store the commands, receivedmessages, and generated messages.

The computer readable recording medium is any data storage device thatcan store data which can be thereafter read by a computer system.Examples of the computer readable recording medium include read-onlymemory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes,floppy disks, and optical data storage devices. The computer readablerecording medium can also be distributed over network coupled computersystems so that the computer readable code is stored and executed in adistributed fashion. Alternatively, the exemplary embodiments may beembodied as computer readable transmission media in carrier waves orsignals for transmission over a network, such as the Internet.

What is claimed is:
 1. A method of decoding an image, the methodcomprising: performing entropy-decoding to obtain quantizedtransformation coefficients of at least one transformation unit in acoding unit of the image; performing inverse-quantization andinverse-transformation on the quantized transformation coefficients ofthe at least one transformation unit to obtain residuals; and performinginter prediction for at least one prediction unit in the coding unit togenerate a predictor and restoring the image by using the residuals andthe predictor, wherein, when a prediction mode is an inter predictionmode, not an intra prediction mode, a size of the at least oneprediction unit in the coding unit is determined independently from asize of the at least one transformation unit in the coding unit, theimage is split into a plurality of maximum coding units, a maximumcoding unit, from among the plurality of maximum coding units, ishierarchically split into one or more coding units of depth including atleast one of a current depth and a lower depth, when the splitinformation indicates a split for the current depth, the coding unit ofthe current depth is split into four rectangular coding units of thelower depth, independently from neighboring coding units, and when thesplit information indicates a non-split of the lower depth, the at leastone prediction unit is obtained from the coding unit of the lower depthand the at least one transformation unit is obtained from the codingunit of the lower depth.
 2. The method of claim 1, wherein theperforming inverse-quantization and inverse-transformation comprisesdetermining a size of the at least one transformation unit based onwhether a prediction mode is an inter prediction mode or an intraprediction mode.
 3. The method of claim 1, wherein theinverse-transformation comprises determining the at least onetransformation unit in the coding unit by grouping prediction areas inthe coding unit.